Story/cite-177 There needs to be an import from Crossref #22

PradnyaC11 · 2024-06-13T18:30:26Z

Guidelines for Pull Requests

If you haven't yet read our code review guidelines, please do so, You can find them here.

Please confirm the following by adding an x for each item (turn [ ] into [x]).

I have removed all code style changes that are not necessary (e.g. changing blanks across the whole file that don’t need to be changed, adding empty lines in parts other than your own code)
I am not making any changes to files that don’t have any effect (e.g. imports added that don’t need to be added)
I do not have any sysout statements in my code or commented out code that isn’t needed anymore
I am not reformatting any files in the wrong format or without cause.
I am not changing file encoding or line endings to something else than UTF-8, LF
My pull request does not show an insane amount of files being changed although my ticket only requires a few files being changed
I have added Javadoc/documentation where appropriate
I have added test cases where appropriate
I have explained any part of my code/implementation decisions that is not be self-explanatory

Please provide a brief description of your ticket

The user should be able to search in crossref to references to import. whatever search results from Crossref a user selects should then be imported into citesphere. This could be all search results or just a few or one.

For now, let's put add an entry to the import menu and let the user select which group to import the results into from a dropdown menu.

The importing should be done from Citesphere Importer (not Citesphere), similar how entries from files are imported.

For querying crossref, this library should be used: https://github.com/diging/crossref-connect. However, this is not yet released into maven central, so talk to Julia before picking up this ticket.

... Put ticket description here and add link to ticket ...
https://diging.atlassian.net/browse/CITE-177

Are there any other pull requests that this one depends on?

diging/citesphere-messages#12
diging/citesphere#271

Anything else the reviewer needs to know?

... describe here ...

…rossrefReferenceImportProcessor accordingly

diging-jenkins · 2024-06-13T18:30:29Z

Can one of the admins verify this patch?

jdamerow

unaddressed comments

jdamerow · 2024-10-10T15:58:01Z

Looks fine. Once the Citesphere PR is ready to be tested, I'll deploy this.

jdamerow · 2024-11-26T16:58:30Z

...n/java/edu/asu/diging/citesphere/importer/core/service/parse/iterators/CrossRefIterator.java

+    private ArticleMeta parseArticleMeta(Item item) {
+        ArticleMeta meta = new ArticleMeta();
+        meta.setArticleTitle(item.getTitle().get(0));
+        meta.setShortTitle(String.join(", ", item.getTitle().subList(1, item.getTitle().size())));


That doesn't seem right. Why would the short title be a concatenation of all other titles? If zotero doesn't allow additional titles, then we might just want to drop them for now.

jdamerow · 2024-11-26T16:59:09Z

...n/java/edu/asu/diging/citesphere/importer/core/service/parse/iterators/CrossRefIterator.java

+            contributors.addAll(mapPersonToContributor(item.getTranslator(), ContributionType.TRANSLATOR));
+        }
+        // List of chair
+        if(item.getChair() != null) {


is that all contributor types that crossref knows?

Yes. crossref has 4 types of contributors - authors, editors, translators & chair

jdamerow · 2024-11-26T16:59:26Z

...n/java/edu/asu/diging/citesphere/importer/core/service/parse/iterators/CrossRefIterator.java

+        }
+        meta.setContributors(contributors);
+
+        meta.setAuthorNotesCorrespondence(null);


isn't that the default?

jdamerow · 2024-11-26T17:00:23Z

...n/java/edu/asu/diging/citesphere/importer/core/service/parse/iterators/CrossRefIterator.java

+        meta.setLanguage(item.getLanguage());
+        ReviewInfo review = new ReviewInfo();
+        if (item.getReview() != null) {
+            review.setFullDescription(item.getReview().getCompetingInterestStatement());  


is full description the same as competing interest statement?

jdamerow · 2024-11-26T17:00:55Z

...n/java/edu/asu/diging/citesphere/importer/core/service/parse/iterators/CrossRefIterator.java

+    private Reference mapSingleReference(edu.asu.diging.crossref.model.Reference itemRef) {
+        Reference ref = new Reference();
+        ref.setAuthorString(itemRef.getAuthor());
+        ref.setContributors(null);


same as before, isn't that the default value?

...n/java/edu/asu/diging/citesphere/importer/core/service/parse/iterators/CrossRefIterator.java

jdamerow · 2025-01-09T14:19:02Z

.../main/java/edu/asu/diging/citesphere/importer/core/service/impl/AbstractImportProcessor.java

        itemTypeMapping.put(Publication.NEWS_ITEM, ItemType.NEWSPAPER_ARTICLE);
        itemTypeMapping.put(Publication.PROCEEDINGS_PAPER, ItemType.CONFERENCE_PAPER);
        itemTypeMapping.put(Publication.DOCUMENT, ItemType.DOCUMENT);
+        itemTypeMapping.put(Publication.BOOK, ItemType.BOOK);


duplicate of line 63

jdamerow · 2025-01-09T14:19:28Z

.../main/java/edu/asu/diging/citesphere/importer/core/service/impl/AbstractImportProcessor.java

+        itemTypeMapping.put(Publication.EDITED_BOOK, ItemType.BOOK);
+        itemTypeMapping.put(Publication.PROCEEDINGS_PAPER, ItemType.CONFERENCE_PAPER);
+        itemTypeMapping.put(Publication.DISSERTATION, ItemType.THESIS);
+        itemTypeMapping.put(Publication.BOOK_CHAPTER, ItemType.BOOK_SECTION);


.../main/java/edu/asu/diging/citesphere/importer/core/service/impl/AbstractImportProcessor.java

jdamerow · 2025-01-09T14:22:03Z

...main/java/edu/asu/diging/citesphere/importer/core/service/parse/crossref/CrossRefParser.java

@@ -0,0 +1,157 @@
+package edu.asu.diging.citesphere.importer.core.service.parse.crossref;


should be moved to an impl package

jdamerow · 2025-01-09T14:23:17Z

...main/java/edu/asu/diging/citesphere/importer/core/service/parse/crossref/CrossRefParser.java

+import edu.asu.diging.crossref.model.Person;
+
+@Component
+public class CrossRefParser implements ICrossRefParser {


needs javadoc and tests

jdamerow · 2025-01-21T16:18:30Z

.../java/edu/asu/diging/citesphere/importer/core/service/parse/crossref/CrossRefParserTest.java

+    }
+
+    @Test
+    public void testParseJournalMeta_WithValidData() {


test method names should follow our naming convention (documented in confluence)

...n/java/edu/asu/diging/citesphere/importer/core/service/parse/iterators/CrossRefIterator.java

jdamerow · 2025-05-05T16:55:01Z

...src/main/java/edu/asu/diging/citesphere/importer/core/service/parse/impl/CrossRefParser.java

+    @Override
+    public ContainerMeta parseJournalMeta(Item item) {
+        ContainerMeta meta = new ContainerMeta();
+        meta.setContainerTitle(item.getContainerTitle().get(0));


Are there more than one title?

Yes, crossref API is returning list of strings for title

jdamerow · 2025-05-05T16:56:28Z

...src/main/java/edu/asu/diging/citesphere/importer/core/service/parse/impl/CrossRefParser.java

+    @Override
+    public ArticleMeta parseArticleMeta(Item item) {
+        ArticleMeta meta = new ArticleMeta();
+        meta.setArticleTitle(item.getTitle().get(0));


same here, are there more than one?

Yes, crossref API is returning list of strings for title

there should be a comment here

jdamerow · 2025-05-05T16:57:54Z

...src/main/java/edu/asu/diging/citesphere/importer/core/service/parse/impl/CrossRefParser.java

+        if(dateParts != null) {
+            publicationDate.setPublicationDate(dateParts.get(2).toString());
+            publicationDate.setPublicationMonth(dateParts.get(1).toString());
+            publicationDate.setPublicationYear(dateParts.get(0).toString());


are all of these always set or could date be null for example?

publicationDate will be null, only if the published date is missing. That was case for some citations. Hance, the check for dateParts is added.

I mean, can dateParts.get(2) return null? If so, toString() would throw an exception I believe?

jdamerow · 2025-05-05T20:51:41Z

Make it so, Jenkins.

jdamerow · 2025-05-05T20:53:50Z

Make it so, Jenkins.

jdamerow · 2025-05-05T20:56:41Z

Make it so, Jenkins.

jdamerow · 2025-05-05T20:58:13Z

I get compilation errors on Jenkins. I think there are some dependencies missing that are needed for the upgrade. Also, can you please update the Java version in the pom to 11?

jdamerow and others added 22 commits July 7, 2022 16:57

[CITE-177] started on processor to import from crossref

0106646

[CITE-177] Trying to add functionality for starting the import

656fab7

[CITE-177] Trying to add functionality for picking the job.

a82d996

[CITE-177] Adding function for starting the import

d3aa18f

[CITE-177] Trying to add crossref import functionality

1710f06

[CITE-177] Trying to create Iterator for Crossref

10d8372

[CITE-177] Iterator changes.

27ee091

[CITE-177] Updating the Iterator and ImportProcessor

f0c4b61

[CITE-177] Adding iterator

58f195c

Adding iterator.

a2d4979

[CITE-177] Correcting the iterator

9616319

[CITE-177] Updating the Crossref Iterator

e3856ea

[CITE-177] CrossrefIterator and identifier

8c4545b

[CITE-177] Resolved error CrossrefReferenceImportProcessor class

c2bfbd0

[CITE-177] Added CrossRef types to CrossRefPublication, and updated C…

74bcd5f

…rossrefReferenceImportProcessor accordingly

[CITE-177] Updated CrossRefIterator

7ecdfd0

[CITE-177] Updated CrossRefIterator

41fed60

[CITE-177] updated generateJson method of JsonGenerationService

7f097df

[CITE-177] Updated itemTypeMapping in crossref import processer

46c2904

[CITE-177] Added more mapping in CrossRefInportProcessor

e36ff3d

[CITE-177] Udpated CrossRefIterator for typeMap and iterator logic

1c94f97

[CITE-177] Added test cases for CrossrefReferenceImportProcessor

bed4968

PradnyaC11 changed the title ~~Story/cite 177~~ Story/cite-177 There needs to be an import from Crossref Jun 13, 2024

PradnyaC11 added 2 commits June 13, 2024 11:38

[CITE-177] Refactoring code to fix issues.

e48be13

[CITE-177] Renamed file to remove unwanted file commit

c086f44

PradnyaC11 requested a review from jdamerow June 13, 2024 18:54

PradnyaC11 mentioned this pull request Jun 13, 2024

Story/cite 177: There needs to be an import from Crossref diging/citesphere#271

Open

9 tasks

PradnyaC11 marked this pull request as draft June 13, 2024 21:11

[CITE-177] Updated CrossRefIterator.java

287ffed

PradnyaC11 reopened this Sep 5, 2024

jdamerow requested changes Sep 6, 2024

View reviewed changes

jdamerow closed this Sep 6, 2024

[CITE-177] Addressed PR comments

b7ded34

PradnyaC11 reopened this Sep 20, 2024

jdamerow requested changes Nov 26, 2024

View reviewed changes

jdamerow closed this Nov 26, 2024

[CITE-177] Addressed PR comments

646c333

PradnyaC11 reopened this Dec 9, 2024

PradnyaC11 requested a review from jdamerow December 9, 2024 23:32

[CITE-177] Resolved code factor issues

02bfdf3

jdamerow requested changes Jan 9, 2025

View reviewed changes

jdamerow closed this Jan 9, 2025

[CITE-177] Addressed PR comments

2575439

PradnyaC11 reopened this Jan 14, 2025

PradnyaC11 requested a review from jdamerow January 14, 2025 23:25

jdamerow requested changes Jan 21, 2025

View reviewed changes

jdamerow closed this Jan 21, 2025

[CITE-177] Addressed PR comments

e8a1876

PradnyaC11 reopened this Mar 27, 2025

jdamerow requested changes May 5, 2025

View reviewed changes

jdamerow closed this May 5, 2025

[CITE-177] Updated dependency versions

f32f4c0

PradnyaC11 reopened this May 6, 2025

[CITE-177] Added javax.annotation dependency

3c80497

		@@ -0,0 +1,157 @@
		package edu.asu.diging.citesphere.importer.core.service.parse.crossref;

Story/cite-177 There needs to be an import from Crossref #22

Are you sure you want to change the base?

Story/cite-177 There needs to be an import from Crossref #22

Conversation

PradnyaC11 commented Jun 13, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Guidelines for Pull Requests

Please provide a brief description of your ticket

Are there any other pull requests that this one depends on?

Anything else the reviewer needs to know?

Uh oh!

diging-jenkins commented Jun 13, 2024

Uh oh!

jdamerow left a comment

Choose a reason for hiding this comment

Uh oh!

jdamerow commented Oct 10, 2024

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jdamerow commented May 5, 2025

Uh oh!

jdamerow commented May 5, 2025

Uh oh!

jdamerow commented May 5, 2025

Uh oh!

jdamerow commented May 5, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

PradnyaC11 commented Jun 13, 2024 •

edited

Loading